Papers with fine-grained evaluation of
MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal large language models (MLLMs) exhibit significant limitations when extracting essential information and reasoned properties from diagrams and performing complex reasoning based on these visual inputs. |
| Approach: | They propose a benchmark that provides a fine-grained evaluation of MLLMs’ perception and reasoning capabilities. |
| Outcome: | The proposed benchmark shows that existing MLLMs exhibit limitations when extracting essential information and reasoned properties from diagrams and performing complex reasoning based on these visual inputs. |
Content Differences in Syntactic and Semantic Representation (N19-1)
Copied to clipboard
| Challenge: | Syntactic analysis plays an important role in semantic parsing, but the nature of this role remains a topic of ongoing debate. |
| Approach: | They propose to use Universal Dependencies and UCCA as test cases to compare syntactic and semantic schemes. |
| Outcome: | The proposed comparison methodology can be used for fine-grained evaluation of UCCA parsing, highlighting both challenges and potential sources for improvement. |
Building an English-Chinese Parallel Corpus Annotated with Sub-sentential Translation Techniques (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study shows that human translators often resort to different non-literal translation techniques besides literal translation . however, they receive less attention in developing natural language processing (NLP) applications. |
| Approach: | They propose to have a better semantic control of extracting paraphrases from bilingual parallel corpora. |
| Outcome: | The proposed method can automatically recognize different non-literal translation techniques . the results confirm the hypothesis of the proposed method . |